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Abstract— Requirements verification of a large flight system is 
a challenge. This paper describes the approach to verification 
of the Origins, Spectral Interpretation, Resource Identification, 
Security-Regolith Explorer (OSIRIS-REx) system 
requirements. It also captures lessons learned along the way 
from the systems engineers embroiled in this process. This 
paper begins with an overview of the mission and science 
objectives as well as the project requirements verification 
program strategy. A description of the requirements flow down 
is presented including an implementation for managing the 
thousands of program and element level requirements and 
associated verification data. This paper discusses both successes 
and methods to improve the managing of these data across 
multiple organizational interfaces. The team’s risk-based 
approach to verifying system requirements at multiple levels of 
assembly is presented using examples from work at instrument, 
spacecraft, and ground segment levels. A discussion of system 
end-to-end testing limitations and their impacts to the 
verification program is included. Finally, this paper describes 
lessons learned during the execution of the verification program 
across multiple government and commercial organizations. 
These lessons and perspectives can be valuable to all space 
systems engineers developing a large NASA space mission. 
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1. INTRODUCTION 


This paper begins with an overview of the mission in Section 
2. Section 3 describes the approach to the requirements and 
verification process across the project. Section 4 and Section 
5 describe the verification process and challenges at the 
spacecraft and instrument levels, using Electromagnetic 
Interference and Electromagnetic Compatibility (EMI/EMC) 
testing as an example of the relationship of the verification 
process at all levels. Section 6 lists the lessons learned 
throughout the process followed by conclusions in Section 7. 
The lessons learned in Section 6 are cross-referenced 
throughout the text using the “LL-#” format to support the 
discussion. The reader is encouraged to refer to these lessons 
throughout the paper to gain more insight into the authors’ 
approach to the requirements verification and validation 
(RVV) process. 


This paper describes a small portion of the work performed 
by several individuals and teams throughout the OSIRIS-REx 
project. For further details, the mission and other systems 
engineering challenges are described in references [1] 
through [4]. 


2. MISSION OVERVIEW 


OSIRIS-REx launched on an Atlas V rocket from the Cape 
Canaveral Air Force Station in Florida on Sept. 8, 2016 at 
7:05 p.m. EDT. It performs an Earth-gravity assist one year 
after launch, then a year later, after traveling for two years, 
the spacecraft will begin its approach to the asteroid Bennu 
in August of 2018. The asteroid is likely to represent a 
snapshot of the early days of our solar system, as well as 
contain a rich supply of carbon, a key element in the organic 
molecules necessary for life. The planned activities at Bennu, 


which will take over two years to complete, are motivated by 
five mission science objectives: 


* Origins: Return and analyze an asteroid sample 


¢ Spectral Interpretation: Provide ground truth or direct 
observations for telescopic data of the entire asteroid 
population 


¢ Resource Identification: Map the chemistry and mineralogy 
of a primitive carbon rich asteroid 


¢ Security: Measure the effect of sunlight to change the orbit 
of a small asteroid, known as the Yarkovsky effect — the 
slight push created when the asteroid absorbs sunlight and re- 
emits that heat as infrared radiation 


¢ Regolith Explorer: Document the regolith (layer of loose, 
outer material) at the sampling site at scales down to the sub- 
centimeter 


After arriving at Bennu in 2018, OSIRIS-REx will begin a 
comprehensive surface mapping campaign using a variety of 
instruments to study the asteroid. OSIRIS-REx will globally 
map Bennu’s surface using optical cameras and a laser 
altimeter. The spacecraft will also use optical, infrared and 
thermal emission spectrometers to generate mineral, organic, 
and thermal emission spectral maps and local spectral 
information of candidate sample sites. The team will use the 
maps and other information gathered by OSIRIS-REx to 
select a location on the asteroid where the spacecraft will 
collect a sample. Once the candidate sample site is selected, 
OSIRIS-REx will approach, but not land on, Bennu. Instead 
of landing, the spacecraft will extend a robotic arm called the 
Touch-and-Go Sample Acquisition Mechanism (TAGSAM) 
to retrieve asteroid samples for analysis. The team intends to 
obtain a sample of at least 2 ounces (60 grams) and as much 
as 4.4 pounds (2 kilograms). The sample will be stored in a 
canister inside the Sample Return Capsule (SRC) as the 
spacecraft travels back to Earth. 


Upon completion of its investigation and sample collection 
of Bennu, OSIRIS-REx will begin its 2.5-year return journey 
to Earth in March 2021. As it approaches Earth, a final course 
correction will set OSIRIS-REx on course to release the 
sample return capsule for a parachute landing at the Utah Test 
and Training Range (UTTR) in Tooele County, Utah, in 
September 2023. The SRC will land at UTTR and will then 
be transported to the Astromaterials Acquisition and Curation 
Office at Johnson Space Center, in Houston, Texas, for 
storage and sample examination. [5] 


NASA’s Goddard Space Flight Center in Greenbelt, 
Maryland, provides overall mission management, systems 
engineering, navigation, and safety and mission assurance for 
OSIRIS-REx. Dante Lauretta is the mission’s principal 
investigator at the University of Arizona. Lockheed Martin 
Space Systems in Denver built the spacecraft and provides 
mission operations. [5] 


3. PROJECT LEVEL 


The OSIRIS-REx mission architecture is comprised of the 
Flight System, Ground System, and Launch Vehicle 
segments. A sample handling segment manages the sample 
following the return to earth in 2023. These segments consist 
of the mission elements shown in Figure 1. The 
requirements are structured to specify the functions, 
performance and interfaces among these mission elements. 


As shown in Figure 1, the project requirements flow-down 
from the Level 1 science and program requirements. The 
Level | science and program requirements were approved by 
NASA Headquarters and the New Frontiers Program Office. 
The Level 1 science requirements were divided into baseline 
and threshold requirements. The baseline mission for 
OSIRIS-REx provides pristine samples of carbonaceous 
asteroid regolith with detailed geologic context. The 
threshold mission does not satisfy these baseline 
requirements yet still provides samples of carbonaceous 
asteroid regolith. The project requirements were 
decomposed to satisfy the baseline mission; however, the 
flight system architecture needed to perform the baseline 
mission is not fully single fault tolerant (single string 
spectrometers). 


The decomposition and flow-down of requirements from 
Level 1 to the element level requirements are shown in 
Figure 1. The Mission Requirements Document (MRD) at 
Level 2 is a project-level document generated by the project 
systems engineering (PSE) team with support from the 
Principle Investigator (PI) and from the element leads. This 
document houses the requirements needed to satisfy the 
Level 1 science requirements and objectives. The 
Environmental Requirements Document (ERD) is held at 
Level 2, since it applies to both the spacecraft and the 
instruments. The ERD is generated by the spacecraft 
provider with project input. The Mission Assurance 
Requirements (MAR) at Level 2 applies across the system. 
The Safety and Mission Assurance team is responsible for 
generating this document and verifying the requirements in 
it. The elements are responsible for tracing their allocated 
Level 2 requirements and developing their Level 3 
requirements. The project approved the Level 3 requirements 
to ensure compliance with the Level 2 requirements. 
Additionally, the project approved the key interface 
requirements documents among the elements (including 
spacecraft-to-instruments, launch vehicle-to-spacecraft). 


All Level 1, 2, and 3 requirements were managed in the 
project DOORS database at Lockheed Martin. Additionally, 
all verification information was documented in the DOORS 
database. While security restrictions limited the users, this 
centralized database of requirements and verification data 
was important for communication across the project 
interfaces and organizations. (LL-1 and LL-2; references to 
lessons learned can be found in Section 6). 


Level 1 


Level 2 
(Project-Developed, 
Project-Approved) 


Level 3 
(Element- 
Developed, 
Project- 
Approved) 


Level 4 
(Element- 
Controlled) 


Figure 1 - Requirements Flow-down 


The PSE team and verification and validation (V&V) team 
were responsible for verifying the Level 2 requirements. The 
elements were responsible for verifying the Level 3 
requirements and all children. The project concurred on the 
verification of all Level 3 requirements. This approach of 
project concurrence of the top level element requirements 
allowed insight into the element verification process and 
added rigor to the entire requirement verification process. 
This insight and rigor ensured that the intent of the Level 2 
requirements were satisfied and the ensured efficient 
verification of the Level | and 2 requirements. 


The cornerstones of the verification and validation program 
are the application of both a design reference mission (DRM) 
and design reference asteroid (DRA) at Level 2. The design 
reference mission documented the concept of operations for 
the complex mission. The DRM was the basis for technical 
performance measures such as data volume and the source of 
scenarios for the V&V program. The DRA used input from 
the science team to specify Bennu parameters used 
throughout the V&V program. These reference design 
documents also provided key input to the Level 2 mission and 
environmental requirements. 


The primary objective of the verification program was to 
mitigate risk to the project during development up to launch. 
To achieve this objective, the project V&V team performed 
several V&V activities throughout development to ensure 
complete and efficient verification of the requirements. 
Requirements validation activities began prior to preliminary 
design review (PDR). These included assessments of the 
flow-down from the MRD into the element requirements. 
This exercise revealed and corrected disconnects in the 
requirements and in the verification plans early in 
development. This saved time and money during the critical 


development phases and was important since most of the 
Level 2 MRD requirements were verified by rolling up the 
lower level verifications. (LL-3) 


Additionally, each Level 1, 2, and 3 requirement was given 
two owners prior to PDR: a project V&V engineer and a 
project subject matter expert (SME). This “two-man rule” 
ensured that verification plans, events, and evidence were 
assessed for both technical and V&V criteria. These 
requirement owners were involved in all phases of 
requirement verification beginning with planning and ending 
with concurrence on the verification documentation (see 
Figure 2). (LL-4) 
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Figure 2 - Project SME and V&V engineer approach to 
verification 


Between PDR and CDR, the project V&V team put 
significant effort into the planning and execution phases of 
the verification process. Questions and disconnects were 
resolved across the project in monthly V&V working groups 
and during weekly meetings among the project and elements. 
This effort ensured that the requirements and verification 
plans were understood. (LL-5) 


Additionally, between PDR and CDR, closeout dates were 
integrated into the verification plans as they were solidified. 
The project V&V team integrated these closeout dates into 
the DOORS database and wrote DXL scripts to integrate the 
verification closure dates into a project verification burn-up 
plan. The plan was baselined around CDR and was managed 
monthly to track progress and communicate status to project 
management (see Figure 3). (LL-6) 
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Figure 3 - Project requirement verification burn-up 


The baseline burn-up plan allowed the project V&V team to 
plan resources throughout the verification process. The 
ability to efficiently update the schedules using the DOORS 
DXL scripts also allowed forecasting of the dynamic 
verification schedules. 


After CDR and during execution of a verification event, 
project V&V engineers participated in test readiness reviews, 
reviewed test procedures, and participated in testing. This 
insight by the project V&V engineers expedited the closeout 
documentation. It also refined the requirements verified in 
the testing and in some cases the testing procedure. (LL-7) 


Project SMEs participated in testing for key tests within their 
disciplines. This insight ensured that tests were executed to 
project and NASA requirements and also expedited any 
needed deviations or waivers. For example, the project EMI 
engineer/SME supported execution of both the spacecraft and 
instrument level EMI/EMC testing. This approach enabled 
quick disposition of exceedances and consultation of any 
testing issue or nonconformance. (LL-4) 


Following verification event execution and documentation by 
the elements, the element teams submitted verification 
reports to the project V&V engineers for concurrence and 
requirement closeout. Every effort was made to streamline 
the concurrence process while making sure all stakeholders 
had input on the review process. The typical flow started 
with an initial assessment by the project V&V engineer for 
completeness: evidence clearly identified, applicable 
verification method, supporting data/documentation 
attached, etc. Due to the large number of organizations 
across the project, no particular format or template was 
required for the verification evidence, which lessened the 
burden on the element teams. As-run procedures, raw test 
results, or test acceptance presentation packages were often 


accepted as evidence. This approach required flexibility at 
the project level and good planning and communication 
between the project and elements, but it reduced the overhead 
for the concurrence process. After the evidence passed the 
initial assessment, an official review was conducted by the 
predetermined project SME. For example, requirements 
verified by an instrument would be reviewed by the Payload 
Office Instrument System Engineer while spacecraft 
requirements were reviewed by the PSE Spacecraft Systems 
Engineer. In some instances, the SME delegated the review 
to project discipline experts (e.g. mechanical for Vibration 
test results, software systems engineer for flight software). 
The requirement was closed when both the project V&V 
engineer and SME concurred that the submitted evidence was 
sufficient to verify the requirement. 


As stated above, the primary goal of the verification program 
was to mitigate risk to the project during development up to 
launch. Consequently, the project V&V team focused on 
verification of flight hardware. As a part of the risk 
mitigation, the primary objective was to close all 
requirements by launch. Additionally, the project V&V team 
established intermediate goals to close all element 
requirements at the time of the element pre-ship review 
(PSR). This strategy was developed to ensure that there was 
low risk to integrating the instruments to the spacecraft and 
low risk to ship the flight system away from the Lockheed 
Martin (LM) integration facility. 


It is inevitable on a large complex mission like OSIRIS-REx 
that some verifications diverge from the plan created around 
the CDR timeframe. As shown in Figure 3, the actual 
verification closures fell behind the original baseline plan. In 
most instances, these delays were the result of delays in 
hardware delivery schedules. In some cases, the pre-ship 
reviews were delayed (OLA, REXIS) and in other cases the 
documentation was delayed to ensure hardware delivery 
dates were met. Consequently, the project V&V team 
developed a risk based approach to prioritize late 
verifications. The approach was also used during Assembly, 
Test, and Launch Operations (ATLO) to make sure that any 
risks to hardware or schedule were captured, evaluated, and 
mitigated to the extent possible. Figure 4 shows our 
adaptation to the typical 5x5 risk matrix. 


This risk management approach to verification met the 
primary goal of the verification program even with delays in 
the closeout documentation. Additionally, the project insight 
and proactive approach to the verification process made the 
risk of open requirements at the PSRs very low. Early project 
communication to the flight elements regarding the 
assessment of open verification items for risk was essential. 
The process primarily established a clear plan and set 
expectations for both teams. This allowed the flight element 
teams to focus on hardware delivery and documentation 
identified as required prior to PSR and significantly reducing 
V&V overhead and miscommunications. Depending on the 
organization, practical implementation included weekly 
telecons or technical interchange meetings. This approach 
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Figure 4 - Open verification risk assessment 


resulted in the instruments and spacecraft being shipped on 
schedule with only low risk open requirements (LL-8). This 
risk management approach to verification was proven 
effective throughout the assembly, test, and launch 
operations (ATLO) program. The next level of integration 
operations and testing that followed hardware delivery 
revealed no significant issues with interfaces and higher level 
testing. Additionally, the project experienced no significant 
delays in the development schedule throughout the ATLO 
program. Finally, all Flight System and ERD requirements 
were fully verified by the elements and concurred by the 
project at launch. 


The Level 1 and Level 2 requirements were the last to be 
closed because these were largely verified using the lower 
level verifications. Delays in the development of the ground 
system past the baseline plan and beyond launch caused 
several ground element requirements and the parent 
requirements to be open at launch. The project V&V team 
again used a systematic risk management approach to 
minimize the risk due to these open requirements. The open 
requirements were assessed and prioritized to focus on the 
requirements needed for launch and early operations. All 
requirements needed for this phase were closed prior to 
launch. Additionally, the project V&V team worked with the 
science team to assess the project and mission compliance 
with each Level 2 science MRD requirement. These science 
team assessments were performed by science team members 
and science working groups (Altimetry, Spectral, etc). These 
assessments included the flight and ground system as-built 
performance, DRM observations, and planned science data 
processing and analysis tools. These assessments were 
documented and peer-reviewed by the Principal 
Investigator’s (PI) office. They revealed liens in 
documentation and deficiencies in the DRM. These liens 
were primarily against observation plans and science data 
processing tools and were not against flight system. In many 
cases the liens were addressed by updates in the SPOC to 
science team interface documentation. All other liens were 
given a closure plan by the PI office and assessed for impacts 
to the requirements verification. Nearly all corrective actions 
didn’t impact the requirement verification and were not 
required until Bennu operations. Therefore; most will be 


resolved as part of the Phase E trades study and change 
management processes. These science team and project 
V&V assessments provided confidence that the Level 1 
science objectives and Level 2 MRD requirements were 
satisfied despite the open requirements at launch. 


This comprehensive approach to verification across the 
project mitigated risk to the mission. Additionally, the 
proactive approach to verification resolved disconnects early 
and eased the process of verification at the element levels. 


4. SPACECRAFT LEVEL 


The OSIRIS-REx Flight System consists of the Spacecraft 
bus, the Sample Acquisition and Return Assembly (SARA), 
and the instruments. The SARA subsystem consists of the 
TAGSAM and the SRC. The OSIRIS-REx Spacecraft bus is 
broken down into eleven subsystems: Command & Data 
Handling (CDH), Electrical Power System (EPS), Flight 
Software (FSW), Guidance Navigation & Control (GNC), 
Harness (HAR), Mechanisms (MECH), Natural Feature 
Tracking (NFT), Propulsion (PROP), Structures (STR), 
Telecommunications (COMM), and Thermal (TCS). LM was 
responsible for the Spacecraft bus and SARA subsystem. The 
instruments are integrated by LM to form the Flight System. 


At the Spacecraft level (level 3), there were two main 
requirements documents to verify- the Spacecraft 
Specification and the Spacecraft to Payloads Interface 
Requirements Control Document (IRCD). The Spacecraft to 
Payload IRCD ensures the instruments will interface with the 
Spacecraft correctly. At the subsystem level (level 4), each 
subsystem has a requirements document to verify. At the 
component level (level 5), each component within a 
subsystem has a requirements document to verify. To save 
time, requirements from heritage programs were used as the 
starting point for the OSIRIS-REx specifications. Thorough 
reviews of the requirements, their applicability to OSIRIS- 
REx, and the planned verification events were performed to 
form the final specifications. (LL-9) 
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Figure 5 -- OSIRIS-REx Spacecraft Requirements Documents 


The two level 3 requirements documents shown in Figure 5 
were closed by LM with concurrence on all requirements by 
GSFC. The level 4 requirements were closed by LM with 
concurrence by GSFC only on specified “key requirements”. 
Weekly meetings were held between the LM and GSFC V&V 
teams to confirm verification expectations, provide 
verification statuses, and answer any verification questions. 
(LL-10) Requirement closure status was tracked in DOORS 
for level 3 and level 4 requirements. The number of 
requirements for each level 3 and level 4 specification is 
shown in Table 1. The level 5 requirements were verified by 
LM or the component vendor and compliance was indicated 
in a verification matrix reviewed upon formal component 
acceptance. 


The level 4 requirements were verified by a combination of 
test, analysis, inspection, and demonstration at both the 
subsystem and system level. The goal is to close requirements 
as early as possible, if adequate verification exists. Therefore, 
subsystem-level verification was used for most level 4 
verifications, except for requirements that required 
interaction with other subsystems/instruments on the 
Spacecraft or verification that required specialized testing 
only available at the system-level. The level 3 requirements 
were verified by a combination of test, analysis, inspection, 
and demonstration at the system-level or a roll-up of 


subsystem level verification, if the subsystem verification is 
sufficient to verify the system-level requirement. 


The Spacecraft and subsystem-level verification timelines 
were created between PDR and CDR and incorporated into 
Figure 3. The V&V report forecasted due dates were placed 
onto subsystem schedules to ensure weekly visibility. (LL- 
11) Viewing the subsystem schedules also allowed the V&V 
team to easily keep up-to-date on status, with minimal 
disturbance to the subsystem teams. The V&V team could 
provide support or request additional V&V resources only 
when required, rather than constantly asking for progress 
updates. (LL-12 and LL-13) The LM formal verification 
report closure process was established after CDR, and 
requirement closure began shortly thereafter. (LL-14) 


A multitude of system-level tests were performed to verify 
spacecraft end-to-end performance. One series of tests is the 
System Verification Tests (SVTs). SVTs test near-real 
mission sequences within the limitations of ACS sensor 
stimulation and limited life hardware cycling. SVTs utilize 
flight operations products to verify successful spacecraft 
execution of all planned mission events and operations for 
each of the Spacecraft’s mission phases: Launch, Outbound 
Cruise, Science, Return Cruise, and Earth Return. A Deep 
Space Network (DSN) test verifies the system interfaces and 
the compatibility of the spacecraft X-band telecom system 


with the DSN Compatibility Test Trailer’s transmitter, 
receiver and data systems. The DSN test also validates 
operating procedures that will be used during the OSIRIS- 
REx mission. Environmental tests, including Sine Vibration, 
Acoustics, Modal Survey, LV Clampband Shock, EMI/EMC, 
and Thermal Vacuum expose the spacecraft to environments 
it will experience during launch and throughout the mission. 
These tests ensure the spacecraft will operate successfully 
throughout the harsh environments of space. The ground 
systems organization also perform Ground Readiness Tests 
and Mission Readiness Tests which validate the Science 
Processing and Operation Center’s (SPOC) ability to 
generate commands, communicate with the spacecraft, and 
display spacecraft telemetry. 


Table 1 — OSIRIS-REx Spacecraft Requirement Counts 


# of Requirements 
Requiring GSFC 
Review 


Requirement # of 
Requirements 


Document 


SCSPEC 409 409 


CDH 322 
EPS 

FSW 

GNC 


The verification team was included in test planning meetings 
and reviews to ensure the spacecraft was in an appropriate 
configuration for the test and the test was gathering the 
appropriate information in order to successfully verify 
requirements. Several of these system-level tests are 
performed multiple times to catch and correct any issues, so 
it was important for the requirement verification to occur at 
the appropriate test run. Additionally, tests are constrained by 
the practicalities involved with testing in a safe manner on 
Earth. A test-like-you-fly philosophy is used throughout the 
process of building and testing OSIRIS-REx, but some 
exceptions are required. Some exceptions required for 
OSIRIS-REx included: electrical ground support equipment 
connected for monitoring spacecraft status and simulating the 
sun and stars, test cables installed for antenna communication 


instead of open-air communication for RF safety/interference 
concerns, Earth’s gravity limiting the motion of solar array 
gimbals and TAGSAM arm movements, and items like 
frangibolts and thrusters unable to be fired during system- 
level testing. Test-like-you-fly exceptions were documented 
and assigned a risk rating. The verification team worked with 
systems engineering until all exceptions were low risk. This 
included adding additional analyses and tests to ensure 
requirements were examined in as flight-like a configuration 
as possible. 


A verification event that was a system-level test with 
incorporation of information from lower-level testing was the 
OSIRIS-REx EMI/EMC test. All electronic components and 
instruments on OSIRIS-REx completed EMI/EMC testing at 
the component level. The emissions and susceptibilities 
found at the component level helped shape the system-level 
test. Two test data gathering locations were needed to get full 
coverage of the entire Spacecraft. Since a component in the 
Telecom subsystem showed minor susceptibilities and a 
component in the C&DH subsystem had emissions close to 
the limits (and these two components were on opposite sides 
of the Spacecraft from each other), these areas were chosen 
as the radiated emissions and radiated susceptibility test 
antenna locations. Figure 6 shows a top-view of the 
Spacecraft with numbers “1” and “2” noting approximate test 
antenna positions. Additionally, component and instrument 
level testing results were used to narrow the frequency bands 
of testing at the system-level test. Low frequencies where no 
radiated emissions or susceptibilities of concern were found 
could be eliminated from the system-level test, saving a 
significant amount of test time. 


Figure 6 — OSIRIS-REx EMI/EMC Test Antenna 
Locations 


The system-level EMI/EMC testing was the verification 
event for 8 Spacecraft Specification requirements. These 


requirements were all related to compatibility with the 
Spacecraft receivers and transmitters, the Launch Vehicle 
receivers and transmitters, and the Launch Site (Kennedy 
Space Center) environment. Mission operating scenarios 
were created for a “Launch Mode”, “Science Survey Mode” 
(performing detailed survey of the asteroid Bennu), and a 
“Science TAG Mode” (performing the sample collection). 
Appropriate combinations of Spacecraft components and 
instruments were powered in each mode, while data was 
collected for radiated emissions and radiated susceptibility. 
Operating modes from the component and instrument-level 
testing were also utilized to help choose appropriate “noisy 
mode” (creating the highest emissions) and “susceptible 
mode” (powering the most sensitive electronics) for use in 
the system-level testing. The specification and optimization 
of these operating modes were the results of numerous 
meetings with experts on each of the different Spacecraft 
components (LL-17). 


During testing, radiated emissions at one location and 
polarization were initially above the required launch vehicle 
limit. Intimate knowledge of the test requirements, however, 
led to the discovery that the limit applied at a farther 
separation distance than the test setup specified. Per the Atlas 
V Launch Services User Guide [8] (and the well-written 
OSIRIS-REx requirement), the OSIRIS-REx Spacecraft 
cannot have emissions above 39 dBuV/m from 1500-1650 
MHz when measured at the top of the Centaur Forward 
Adaptor (CFA). When the OSIRIS-REx test antenna was 
moved to a location more representative of the CFA, all 
emissions met LV limits. (LL-18) Note the actual OSIRIS- 
REx radiated emissions limits for the launch vehicle are 
slightly different than those shown in Figure 7 
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figure has been cleared for public release under OSR approval 10-S-0689, 
date: 16 February 2010 and has been placed in the public domain. 


Figure 7 — Spacecraft Generated EMC Environment 
Limitation 


The EMI/EMC system-level test successfully completed in 
five days. All radiated emissions and radiated susceptibility 
tests passed with no issues. To complete the verification 
effort, a test report was written by LM to detail the results of 
testing and the requirements verified. The report was 
reviewed and signed by a representative for each subsystem 


and one for the system at LM. The requirements were then 
marked complete in DOORS and the report was sent to GSFC 
for concurrence with the requirement closure. 


5. INSTRUMENT/OCAMS LEVEL 


The OSIRIS-REx Mission contains a_ sophisticated 
instrument suite required to fully map and characterize the 
asteroid Bennu and all potential sample sites. The instrument 
suite, designed and built by separate partnering institutions, 
is located on the top deck (+Z) of the flight system as shown 
in Figure 8. The instrument suite consists of two 
spectrometers, a thermal emission spectrometer (OTES) built 
by Arizona State University and a visible and infrared 
spectrometer (OVIRS) built by Goddard Space Flight Center 
(GSFC). A laser altimeter (OLA) provided by the Canadian 
Space Agency expands the capabilities of the instrument suite 
along with the Massachusetts Institute of Technology student 
experimental instrument (REXIS), an x-ray spectrometer. To 
ensure the mission’s extensive imaging needs are met, the 
OSIRIS-REx Camera Suite (OCAMS), designed and built by 
the University of Arizona, is a set of three cameras (PolyCam, 
MapCam, and SamCam) with overlapping capabilities, 
including a fully redundant electronics system, to ensure 
asteroid acquisition, mapping, and sample collection 
verification. The three cameras each utilize a common 
detector assembly. 


MAPCAM 


POLYCAM 


Figure 8 — The OSIRIS-Rex Instrument Suite shown on 
the Science Deck of the Spacecraft [6] 


As a camera suite, OCAMS is levied requirements from a 
decomposition of mission requirements (level 2). The 
OCAMS instrument requirements are owned by the OCAMS 
team but controlled by the GSFC Payload Office. Similarly, 
the Instrument to Spacecraft Interface Requirements 
Document is owned by the spacecraft team at LM but 
controlled by the managing authority at GSFC. The typical 
requirements flow down structure presented a unique 
challenge for the OCAMS systems engineering team. The 
OCAMS architecture indicates a single instrument; however, 
the implementation of the system reveals that each of the 
three cameras can be examined and tested independently. 
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Figure 9 -- OCAMS Requirement Tree illustrates the complex requirements structure necessary to implement a 
robust verification program. 


This paradigm drove the design of the requirements and 
verification processes. Thus, the OCAMS level 3 
requirements identified the observing scenarios and the 
performance envelope for each of those observations. The 
level 4 requirements are then defined for camera and 
electronic assemblies to maintain verifiability. Subassembly 
requirements, _ component specifications, software 
requirements, and individual electronic board specifications 
are hosted with the level 5 requirements documents. The 
complex relationships between subsystem and system level 
requirements were recognized early in the design life cycle. 
[4] The requirements flow down structure expanded to 
include links from mission level requirements directly to the 
affected subsystem requirements. This addition enabled 
independent verification while not precluding combined 
verification events which is evident in Figure 9. It also 
provided further visibility into the verification efforts by the 
technical management and systems team at GSFC. 


Well-defined requirements and a closed-loop configuration 
control process provided the foundation for developing a 
robust verification program for the nearly 2,000 OCAMS 
requirements (LL-16). The University-of-Arizona-led 
instrument team was resource limited, so an agile verification 
approach was necessary to adapt to the evolving mission 
design guided by the MRD. The verification efforts started 
with the delivery of the System Verification Plan. This 
document identified the verification method, test set-up, 
assumptions, and essential data necessary to verify each level 
3 requirement. The OCAMS systems engineering team led 


internal working groups to determine calibration 
requirements for test equipment and to identify data 
processing tools that would be utilized or developed to 
achieve confident technical performance measures. This 
created a  requirements-centric culture within the 
development team. The Verification Description Document 
(VDD) was born from this relationship to document 
requirement rationale, verification method, and initial test 
schema or concept for validation. Lacking the infrastructure 
of a large aerospace conglomerate, the VDDs provided the 
framework essential to executing a successful verification 
program. 


Understanding the requirements necessary to establish a fully 
NASA compliant instrument verification program is the first 
step in implementing a successful integration and test 
program. Therefore, OCAMS literally took a page from the 
Marshall Space Flight Center Verification Handbook, which 
provided guidelines and best practices for implementing a 
successful integration and environmental test program [7]. 
Developed with customer involvement, the Verification 
Event Datasheet (VED) was a tailored documentation control 
tool designed to document the state of a requirement 
throughout subsystem build and test and simplify the tracking 
process as the number and complexity of requirements 
expanded (LL-6). It maintained a closed loop process with 
Engineering Change Requests, Problem Failure Reports, 
Special Test Requests, and any potential waivers (LL-15). 
The VED was implemented at the earliest stages of the 
OCAMS build and continued through all environmental tests. 


Thus, verification events could be assigned and tracked as 
soon as procedures were released, allowing for early 
identification of failures, anomalies, or even reductions to 
planned margin. Outside of being a useful tool to track 
verification and identify potential threats, the VED provided 
efficiency at the customer interface. Information was easily 
transferred to the project verification group at GSFC and 
furthermore, the VED provided detailed visibility into the 
status of OCAMS development and test. The usefulness of 
the VED is easily illustrated through the verification process 
in Figure 10. 
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Figure 10 — The OCAMS Verification Process is uniquely 
coupled to development life cycle and customer interface. 


The advantage to such a robust verification process is most 
evident in the instrument EMI/EMC test. The test was 
designed to satisfy 42 OCAMS environmental requirements 
and some lower level requirements that had been deferred to 
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the higher level test to mitigate schedule and resource 
concerns. The strength of the instrument level EMI/EMC 
testing was its completeness, thus allowing the system level 
testing at the spacecraft to be reduced to only higher 
frequency tests of emissions and susceptibility. Recognizing 
the complexity of the OCAMS instrument, a comprehensive 
test that examined each camera as an independent unit was 
deemed most valuable. The test setup was designed to be both 
compliant with level 2 requirements but also contain enough 
flexibility to test OCAMS in all flight-like configurations. A 
simplified block diagram is shown in Figure 11 highlighting 
this configuration. To meet the multi-imaging needs of the 
mission, the three cameras are located in different physical 
locations across the spacecraft science deck. Following the 
“Test Like You Fly” philosophy, a Mechanical Interface 
Control Document (MICD) compliant fixture was fabricated 
to best mimic the portion of the spacecraft deck that OCAMS 
inhabited, which can be seen in Error! Reference source not 
found.. The modes of operations had to be examined next to 
define what detectors, motors, or LEDs would be powered, 
and what would be the implications to other subsystems. It 
was determined that all permutations would be examined for 
susceptibility and emissions except for the case where all 
three imagers are powered. This was excluded because the 
mission requirements never utilized more than two cameras 
at once and lower level requirements prevented such a case 
to prevent the likelihood of OCAMS being in an unsafe state. 
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Figure 11 -- OCAMS Instrument Level EMI/EMC Test Configuration Block Diagram. 


The resulting test cases exercised several different 
operational scenarios for asteroid imaging campaigns. Some 
examples included using MapCam and PolyCam as required 
during approach, reconnaissance and orbital phases. Other 
examples included using MapCam and SamCam in a manner 
consistent with TAG and TAG rehearsals. These flight-like 
scenarios increased the duration repetitions of several of the 
lower frequency scans. During the high frequency X-Band 
scans, PolyCam became the sole imager to examine both 
emissions and susceptibilities because it towered over the 
smaller cameras and also acted as an antenna, being the tallest 
metallic structure on the deck. Utilizing a single camera, each 
scan could be more efficiently examined since the time spent 
changing camera modes was reduced (which could 
conservatively take up to 10 seconds) and more time was 
utilized in either the quiet or noisy configuration. The test was 
successfully completed and approved by the verification 
engineers at GSFC with certification from SME. Any 
potential over-limits identified were dispositioned and noted 
for close examination during spacecraft level EMI/EMC. 


EMI/EMC was performed in such a realistic and flight-like 
fashion, the entire instrument could be assessed in an end-to- 
end manner. From routing cables as they would be on the 
deck to populating a list of idiosyncrasies that would provide 
inputs to the SVTs at the spacecraft level helped to lower risk 
during vehicle integration. This allowed lessons learned to be 
captured early and communicated upstream to other elements 
smoothing the transition from instrument to spacecraft to 
science operations. 
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Figure 12 - OCAMS EMI/EMC Test Setup Flight-like 
Cable and Instrument Configuration 


6. LESSONS LEARNED 


The previous sections contained references to the lesson 

learned shown below. This sections lists those lessons 
learned throughout the development. These lessons are 

LL-1: Creating a centralized database of requirements and 

verification data fosters communication across the 

project interfaces and organizations. However, IT 
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security guidelines must be considered when using 
a central database for management of the 
requirements and verification data across several 
organizations. Stringent controls of the data that 
precludes access limits the access of the team and 
creates bottlenecks in the RVV _ process. 
Additionally, remote access (e.g. Citrix client) 
reduces the performance and usability of interactive 
tools such as DOORS that also creates bottlenecks. 
Verification plans will inevitably change (often), so 
the database for those plans must be designed to be 
user friendly and easy to edit. Also, the database 
needs to be user friendly enough so that all 
stakeholders will frequently use it, rather than 
placing all updating responsibilities on the RVV 
team. 

Early involvement by the project V&V team into the 
element requirements development and verification 
planning enables smooth execution and closeout of 
the requirements verification program. Early 
resolution of disconnects in requirements flow- 
down and verification plans mitigates risk to the 
mission. 


LL-2: 


LL-3: 


LL-4: Collaboration of the project V&V team and the 
project SMEs resolves disconnects in_ the 
requirements and verification plans. It also 


expedites the processing of test deviations and 

waivers and verification documentation. 

Verification process should be defined early, while 

reporting expectations should be contractually 

controlled, to improve efficiency when verification 
efforts inevitably ramp up. The biggest advantage of 
identifying verification efforts during phase A and 

B is enabling proper scope and estimation thus, 

easing the stress of integration and test engineers 

Detailed reviews of verification plans should be held 

as early as possible and with a high level of rigor in 

order to identify verification event gaps and 
subsystem/system boundaries of responsibility. 

Additionally, the high-level milestone verification 

dates should be reviewed with management at/prior 

to CDR to determine if proposed verification dates 
are acceptable to the program. 

Integration of the RVV database (DOORS) with 
other tools such as the test procedure 
development/execution software (Automation 
Framework) was very beneficial to reducing errors 
and increasing efficiency. This integration coupled 
with V&V engineers being involved in test 
readiness reviews and test procedure reviews was 
essential to making sure the right requirements were 
being tested and bought off by the appropriate test, 
and that no requirements were missed. 

LL-8: All requirements are not critical. The verification 
engineer and SME should perform an assessment of 
risk as part of the verification process. A systematic 
risk assessment can help prioritize verification 
activities with other tasks when encountering 
programmatic constraints. 


LL-5: 


LL-6: 


LL-7: 


LL-9: Leveraging heritage requirements from previous 
missions saves a lot of effort in creating 
requirements for a new program. However, care 
must be taken to ensure the requirements carried 
forward are still applicable, and also are verifiable. 
Just because a previous program had a requirement 
that applies to the new program doesn’t mean that 
requirement was well-written or had a 
straightforward verification plan. The requirements 
and verification plan should be supported by the 
heritage verification documentation when using this 
approach 

LL-10: Close coordination between GSFC and LM was 

essential for establishing a set of common 

expectations for the verification process. For 
example, LM considers a requirement bought-off 
when all the linked verification events were 
complete, while GSFC was expecting that all lower- 
level linked requirements to be bought off first. 

Weekly V&V meetings between GSFC and LM 

helped identify gaps in expectations and provided a 

forum for all verification questions or concerns. 

Subsystem verification reports were tracked in the 
subsystem schedule along with other subsystem 
tasks. This ensured the V&V reports (and their due 
dates/percent complete) were discussed weekly in 
planning meetings, so they didn’t slip through the 
cracks and become forgotten. 

LL-12: The V&V team should take time every few weeks 
to sit down with the subsystem leads to discuss 
progress and how the V&V team can help. Creating 
draft reports for subsystems who were running 
behind greatly increased report output (and good 
feelings towards the verification team). These 
meetings also led to meetings with managers to get 
surge support for verification, as needed. 

LL-13: Don’t be afraid to bring verification reports or 
signatures that have remained open for a long time 
to the attention of management. They really can help 
push things to closure! 

LL-14: The verification report closure process should 
ideally be established prior to CDR. Several 
analyses were written for CDR as the final version, 
but were not released as a “verification report”, so 
extra effort was required to revisit the analysis and 
re-release it with the appropriate verified 
requirements noted. 

LL-15: All requirement owning elements should conduct 
weekly or biweekly verification review boards to 
assess buy-off status, or intermediate verification 
product status. This review board should be 
managed by the systems engineering team and 
chaired by the verification engineer. The 
verification review board should be conducted much 
like a CCB or a FRB and open to any and all 
discipline and design engineers who seek insight 
regarding requirements. 

LL-16: The verification process should be closed loop with 
all systems engineering processes. Interlacing 


LL-11: 


12 


verification activities with configuration 
management and quality assurance processes 
presents an opportunity to assess validation of past 
verification events or future verification plans. 


Additionally, the authors also learned several lessons 
throughout the project that are applicable to all systems 
engineers. These include: 

LL-17: Communication is key! Every person on the project 
has different experience and expertise — use that to 
your advantage. It is more time and cost efficient to 
talk to other engineers than to try to know/do 
everything yourself. 

LL-18: MIL-STDs are useful guidelines for test 
configurations, including System-Level EMI/EMC 
testing. Actual program requirements should take 
precedence over the MIL-STD techniques, when 
applicable. 

LL-19: Changes across multiple elements and 
organizations take longer than expected. This 
implies that planning and strategy discussions are 
important to ensure issues are resolved in a timely 
manner. Experience has shown this is much longer 
than making changes and resolving issues at the 
subsystem level. 

LL-20: When an engineering team consists of multiple 
people, great care should be taken to ensure 
everyone knows what the other team members are 
working on. There were several instances where 
V&V work was being somewhat duplicated by team 
members or time savings could have been realized 
by more efficiently distributing data products 
among the team. 

LL-21: Don’t be afraid to respectfully challenge the 
engineering rationale of experienced colleagues and 
subject matter experts. This often presents system 
level learning opportunities that can provide insight 
across multiple interfaces and disciplines. 

LL-22: Identify a project mentor and a professional mentor. 
The project mentor should be someone within your 
project organization who understands your role 
within the context of the project and can provide 
guidance on technical skill growth. The professional 
mentor should be someone who is not directly 
associated with the project but understands the 
industry well and can provide career guidance and 
tips for professional growth. 


7. CONCLUSION 


This paper discussed the lessons learned throughout the 
OSIRIS-REx requirements verification program. The project 
applied a comprehensive, proactive approach to the 
verification of the requirements that was used to mitigate risk 
to the mission. All elements of the flight system were 
successfully integrated throughout ATLO with no significant 
delays. The vehicle successfully launched within seconds of 
its launch window opening. All systems are performing 
nominally as verified during post-launch functional and 


performance tests. The flight system is operating without 
problem. 
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